Papers by Chengzhong Xu Xu
An Anchor-based Relative Position Embedding Method for Cross-Modal Tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Position Embedding (PE) is essential for transformer to capture the sequence ordering of input tokens. |
| Approach: | They propose a unified position embedding method that bridges the semantic gap between modalities and embeds the anchor-based distance to guide computation of cross-attention. |
| Outcome: | The proposed method obtains new SOTA results on a wide range of benchmarks. |